The demand for storage capacity is growing rapidly. As a result, disk storage is at a premium and can be a significant cost to those who need to store a large amount of data. One approach to reducing data storage costs is through data compression. Data compression can be used to store more information per unit of storage (e.g., per megabyte), thus reducing the need for additional storage devices. Various techniques for improving the effective capacity of storage have gained attention. Data compression may be employed in order to reduce the storage space needed to store certain data, and may also be used to reduce data transfer costs.
One useful type of data compression is block compression, where subsets of a data block are compressed separately and individually. Block compression, however, is generally computationally expensive in terms of central processing unit cycles involved in doing the compression and can cause the compressed data to be inefficiently stored. One technique that may be used to alleviate these two problems is to use incremental compression, where the data is incrementally compressed based on the block size so as to improve the storage inefficiencies. Compute resources spent on compression may lead to workload problems on the processor, potentially decreasing performance of the compression and/or slowing down the system as a whole.